Segmentation non supervisée : le cas du mandarin

Unsupervized Word Segmentation In this paper, we present an unsupervised segmentation system tested on Mandarine Chinese. Following Harris’s Hypothesis in Kempe (1999) and Tanaka-Ishii (2005) reformulation, we base our work on the Variation of Branching Entropy. We improve on (Jin et Tanaka-Ishii, 2006) by adding normalization and Viterbi-decoding. This enables us to remove most of the thresholds and parameters from their model and to reach near state-of-the-art results (Wang et al., 2011) with a simpler system. We provide evaluation on different corpora available from the Segmentation bake-off II (Emerson, 2005) and define a more precise topline for the task using cross-trained supervised system available off-the-shelf (Zhang et Clark, 2010; Zhao et Kit, 2008; Huang et Zhao, 2007) MOTS-CLÉS : Apprentissage non-supervisé, segmentation, chinois, mandarin.